\documentclass[12pt]{article}


\usepackage{hyperref}
	\hypersetup{
     colorlinks   = true,
     allcolors    = black
		}

\usepackage[singlelinecheck=false, justification=centering,font={normalsize},labelfont={sc}]{caption}
\usepackage[font={footnotesize},labelfont={rm}]{subcaption}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{geometry}
\usepackage{lscape}
\usepackage{graphicx}
\usepackage[justification=justified,singlelinecheck=false]{caption}

\usepackage[american]{babel}

\usepackage{setspace}

\usepackage{comment}

\usepackage[round]{natbib}
\setcitestyle{aysep={}}
\bibliographystyle{jaere}
\newcommand\cites[1]{\citeauthor{#1}'s\ (\citeyear{#1})}


\setlength{\textfloatsep}{15pt}
\setlength{\floatsep}{30pt}



\title{\large Incentives and Information in Methane Leak Detection and Repair
\footnote{Lewis: Texas A\&M University; Wang: Salesforce; Ravikumar: The University of Texas at Austin. We are grateful for helpful comments from Mark Agerton, Kristina Brecko, Thomas Covert, Brigham Frandsen, Ben Gilbert, Evan Herrnstadt, Ryan Kellogg, and seminar participants at the International Industrial Organization Conference, Association of Environmental and Resource Economists, and Texas A\&M University. Corresponding author is Eric Lewis (ericlewis@tamu.edu)}}

\author{Eric K. Lewis \and  Jiayang (Lyra) Wang \and Arvind P. Ravikumar}

\date{\today}

\begin{document}

\maketitle

\begin{abstract}

Capturing leaked methane can be a win for both firms and the environment. However, leakage volume uncertainty can be a barrier inhibiting leak repair. We study an experiment at oil and gas production sites which randomized whether site operators were informed of methane leakage volumes. At sites with high baseline leakage, we estimate a negative but imprecise effect of information on endline emissions. But at sites with zero measured leakage, giving firms information about methane leakage \emph{increased} emissions at endline. Our results suggest that giving firms news of low leakage disincentivizes maintenance effort, thereby increasing the likelihood of future leaks.

 \end{abstract}
 
 \newpage
 \onehalfspacing



 \section{Introduction}

In many settings, an agent must learn about the nature of a problem before fixing the problem. Therefore, giving the agent information about the problem reduces learning costs and can therefore potentially facilitate fixing the problem. Such information provision can be especially socially desirable when fixing the problem also decreases negative externalities. However, the effect of providing information is theoretically ambiguous and in some cases can make the problem worse. This paper discusses one such setting -- methane leak detection and repair at oil and gas production sites -- and shows how providing information about methane leaks can lead to adverse effects. 


Oil and natural gas production results in 78 million tons of global methane emissions per year, equal to 21\% of total global anthropogenic methane emissions, with approximately \input{Figures/single_numbers/iea_fraction_leaks.tex}\unskip\% of those oil and natural gas production methane emissions coming from unintentional leaks \citep{bib:UNEP_CCAC, bib:iea_methane_tracker_2023_database}. In North America where production has surged due to the fracking revolution, 80\% of the increase in methane emissions from 2001 to 2017 came from the fossil fuel industry \citep{bib:Jackson_2020}. Fixing methane leaks is likely desirable not only from a social perspective but also from a private perspective: From a social perspective, eliminating leaks is almost certainly efficient because methane emitted to the atmosphere has approximately 30 times the global warming potential as CO\textsubscript{2} over a 100 year time period, where CO\textsubscript{2} is the byproduct of burning methane \citep{bib:IPCC_chapter7}. In addition, \citet{bib:gordon} finds that at current US methane leakage rates, the environmental impact of natural gas consumption surpasses that of coal in terms of its contribution to global warming. From a private perspective, leak repair increases revenues, and  \citet{bib:iea_methane_tracker_2023_database} estimates that \input{Figures/single_numbers/iea_fraction_negative_abatement_cost.tex}\unskip\% of upstream leak detection and repair could be done at \emph{negative} abatement costs. The fact that many methane leaks are not repaired is therefore suggestive of additional frictions such as uncertainty about leakage volumes. 

Emissions uncertainty is especially important for methane emissions at upstream oil and gas production sites. Produced natural gas passes through pipes, tanks, and valves where corrosion and weather put wear and tear on components and potentially lead to leaks. Because the location of leaks could be nearly anywhere at the well site and well sites are often remote and large, methane emissions measurement is costly, and especially so if the measurement is to be done frequently and with a low detection threshold.\footnote{As a result of these measurement challenges, official government measures of methane emissions from upstream oil and gas production sites are often either based on firms's self-reported measures or rely on imputation. In the United States, the EPA Green House Gas Reporting Program (GHGRP) imputes methane emissions for oil and gas production sites as a function of production, facility characteristics, and pre-determined emissions factors. This is in stark contrast to data on greenhouse gas emissions from industrial production facilities, where greenhouse gases are emitted largely through well-defined exit points and are measured directly.} Because producing firms do not internalize the negative externalities of methane emissions, producing firms are under incentivized to invest in the efficient level of methane emissions monitoring. 





Therefore, one potential policy solution to methane emission uncertainty is to provide external monitoring of methane leaks and communicate the results from that monitoring to firms. Through externally-provided information on the volume and location of leaks, the firm can better target repair activity without paying the costs of monitoring and learning. However, the effect of providing this information to firms will depend on how much methane is leaking. Intuitively, firms are more likely to be incentivized to fix leaks when the leakage volume is high. In section \ref{sec:model}, we write a simple model where a firm makes leak detection and repair choices: We show that when leakage is high, information provision will weakly decrease future leakage, but that when leakage is low, information provision will weakly \emph{increase} future leakage. This latter effect is driven by the good news of low leakage disincentivizing the firm from doing site visits and maintenance, thereby increasing the likelihood of future leaks.



In section \ref{sec:sum_stats}, we introduce data from a randomized controlled trial (RCT) in the engineering literature by \citet{bib:wang} in which oil and gas production sites in Alberta were randomly assigned to either treatment or control: For both treatment and control sites, the \citet{bib:wang} research team performed leak detection surveys, but only for treatment sites were the results of this leak detection reported to the site operator. The data includes information both on leakage as well as venting, where the latter is intentional methane emissions incorporated into production design such as to power control devises or when there is unexpectedly high methane pressure build up. The researchers then surveyed the sites again at endline, again comprehensively measuring leakage and venting. The focus of \cite{bib:wang} is largely on describing the various components that contribute methane emissions and tracking efficacy of repair, with only a short appendix discussion of the average treatment effect.\footnote{Overall, their data shows that there was a reduction in average emissions for both treatment and control groups, with the reduction at treatment sites being larger than but not statistically different from the reductions at control sites.} Therefore, our paper builds on \citet{bib:wang} by using their data to examine the heterogeneous treatment effects of information provision -- how informing the firm that it had either high or low emissions at baseline -- affects future emissions outcomes. We argue that the experimental setting  of \citet{bib:wang} -- Alberta in 2018 and 2019 -- is a particularly good setting to examine the impact of information. During that time, Alberta had virtually no legal penalties for firms with high methane emissions. Because of this, we argue that the effect of the treatment is driven by firms updating beliefs about how much revenue is lost via leakage, and not driven by changes in the probability nor perceived probability of regulatory penalties.  

In section \ref{sec:treat_effects}, we estimate a nonparametric heterogeneous treatment effects specification where endline emissions depend on baseline leakage and treatment status. We find results matching the intuition of the model: Treatment is associated with a statistically insignificant decrease in leakage when baseline leakage is high but a statistically significant increase in leakage when baseline leakage is low. We include a number of robustness checks for the case where measured baseline leakage is zero: For those sites, we find that information provision leads to approximately a \input{Figures/single_numbers/median_leak_effect_round.tex}kilogram per day increase in leakage at endline and a \input{Figures/single_numbers/median_emiss_effect_round.tex}kilogram per day increase in total emissions (leakage plus venting) at endline for each treated site. Multiplying the latter by the climate cost of methane implies that the information treatment causes approximately \$\input{Figures/single_numbers/median_emiss_effect_DOLLARS.tex}of climate damage per day per treated site.\footnote{In 2023 dollars. Throughout the paper, we follow \citet{bib:interagency} and \citet{bib:agerton_gilbert_upton} and assume a social cost of methane of \$\input{Figures/single_numbers/methane_social_cost_per_kg.tex}\unskip/kg.} 


Then, in section \ref{sec:policy_implications}, we discuss policy implications: Because our results show that giving firms information about baseline leakage has adverse effects on emissions when baseline leakage is low but not high, we examine whether a regulator can better reduce emissions by only informing firms of leakage volumes when leakage exceeds a certain threshold. We show that such a strategy can actually lead to even higher emissions than both never informing and always informing the firm. This effect is driven by how firms update beliefs in the absence of information provision. We also discuss the challenges of implementing efficient methane control given both firm and regulator uncertainty in emissions. For example, we discuss how forthcoming advances in global satellite methane monitoring can potentially \emph{increase} methane emissions at low-emission sites. 


This paper builds on a small economics literature on incentives to reduce methane emissions: \citet{bib:marks} and \citet{bib:dunklewerner_qiu} estimate the marginal abatement cost of methane and examine the impacts of methane taxes and targeted audits plus fees, respectively. \citet{bib:cicala} examines how voluntary self-monitoring coupled with methane taxes can reduce methane emissions from oil and gas production sites. \citet{bib:hausman_muehlenbachs} show that natural gas utilities tend to underinvest in leak repair and overinvest in pipe replacement. \citet{bib:agerton_gilbert_upton} provide an economic framework and outline policy challenges related to venting, leaking, and flaring. In engineering, a growing literature is measuring and characterizing methane leaks from oil and gas operations, including \citet{bib:brandt_heath_cooley}, \citet{bib:chen_etal_2022}, \citet{bib:omara}, and \citet{bib:wang}, where the latter is the source of our data. Our paper is one example of how economics can be used to glean additional insights from data collected in other disciplines. 

This paper contributes to literature on how disclosing information -- and specifically information about one's own type -- affects an agent's actions. \citet{bib:costa_kahn} give households information on their own (and neighbors') electricity consumption; they find that information causes some households to increase consumption and others to decrease consumption. Similarly, \citet{bib:castillo_petrie} give households information about natural gas consumption and find that, without additional conservation incentives, information provision increases natural gas consumption. \citet{bib:camargo} show that test score disclosure increases future test scores at private schools. The mechanism that we highlight -- that revealing good news can disincentivize effort -- is potentially present in any setting where information is revealed to agents, including the effect of receiving health assessments on preventative health care effort, the effect of receiving performance evaluations on employee effort, and the effect of corporate responsibility reports on CEO effort. Complementing \cites{bib:costa_kahn} and \cites{bib:castillo_petrie} work on households, we show that this adverse effect of information can also exist for firms and that it does not rely on behavioral mechanisms such as salience and inattention.

This paper is also related to optimal policy with negative externalities and uncertainty. In contrast with the classic \citet{bib:weitzman} setting with uncertainty about costs and benefits, our setting also has uncertainty about emissions themselves. But unlike nonpoint source pollution problems where firm-level emissions cannot be measured \citep{bib:segerson, bib:braden_segerson, bib:xepapadeas, bib:kotchen_segerson}, site-level methane emissions can be measured although measurement requires costly effort. While costly, such measurement is necessary to implement policies such as Pigouvian taxes and transform the problem into a point source pollution problem. This paper is also related to literature on the environmental consequences of other sources of uncertainty, including uncertainty about future policies \citep{bib:dorsey2019waiting, bib:johnston_parker, bib:gowrisankaran_langer_zhang, bib:chen}. 

 
 \section{Background}
 \label{sec:background}
 
 \subsection{Methane emissions, detection, and repair}
 \label{sec:industrial_background}
 
This paper studies two types of methane emissions from oil and natural gas production: leakage and venting.\footnote{A third type of methane emissions comes from incomplete flaring which is the failure to combust 100\% of the natural gas in flaring operations \citep{bib:agerton_gilbert_upton}. We do not examine flaring in this paper as flaring is not tracked within our data.}  Leakage is the unintentional release of methane. Because methane is a gas, any unintentional failure to close the production infrastructure will result in leakage. As a result, leaks commonly occur at joints and valves where adjoining pieces of metal tubing are connected, as well as in tanks where large volumes of gas are held. In contrast with unintentional leakage, venting is the intentional release of methane through specially designed components. For example, many control and measurement devices are pneumatically powered through natural gas pressure where natural gas is released into the atmosphere after powering the component. While venting is intentional, venting may release more methane than intended, and therefore excess venting can be thought of as another form of leakage. 



There are a variety of technologies available for detecting and measuring leakage and venting emissions. Some of these technologies require on-site visits, including hand-held detectors and optical gas imaging (OGI), the latter being the technology used to measure emissions in this study. These on-site technologies typically have low minimum detection thresholds, are relatively low marginal cost to implement per potential leak, but also require costly on-site visits.\footnote{The marginal cost is largely for labor as the capital costs for equipment are largely a fixed cost. However, costs can vary depending on type of leak: Hand-held detectors require physically standing close to the leak, and therefore are more costly to deploy when emission sources are less accessible, such as at the top of tanks. OGI costs are largely constant regardless of the type of leak because OGI uses a camera and can image emissions even if the emission source is out of physical reach.}  In contrast, remote detection of methane via airplane and satellite does not require costly site visits, but has much higher minimum detection thresholds (with especially high minimum detection thresholds for satellites), and is not necessarily able to distinguish between venting and leakage. New methods for on-site continuous monitoring from towers have high installation costs, were not available at the time of \citet{bib:wang} study, and as of 2024 are still in early stages of commercialization. The absence of technology that can continuously measure emissions accurately at low cost implies that firms have typically had at least some uncertainty about their methane emissions. 

Whether or not site visits are used for leak detection, site visits are necessary for leak repair. Site visits are also necessary for regular site maintenance; such maintenance is important because weather and other shocks cause wear and tear on equipment and therefore increase the likelihood of future leaks. However, site visits are costly, and the non-convexity imposed by site visit fixed costs is an important feature in our model of leak detection and repair in section \ref{sec:model}. 

 \subsection{Regulatory setting}
 \label{sec:regulatory_background}

Our experimental setting -- Alberta in 2018 and 2019 -- was one in which methane emissions policies and enforcement for oil and gas production sites were relatively lax. Alberta's Oil and Gas Conservation Act, as written in 2017-2020, prohibited wasteful operations including ``the escape or the flaring of gas, if it is estimated that, in the public interest and under sound engineering principles and in the light of economics and the risk factor involved, the gas could be gathered, processed if necessary, [and either sold or used]" \citep{bib:alberta_oilandgasconservationact}. To implement this act, the Alberta Energy Regulator (AER) imposed directive 060, which imposed limits on flaring and venting \citep{bib:AER_060_2018}. Specifically, total emissions from venting and flaring exceeding 900 cubic meters per day (604 kg/day) needed to be justified and if possible reduced to lower levels, e.g., through selling the natural gas rather than venting it. In some cases, venting limits were even lower.\footnote{For example, in cases of wells with a high gas to oil ratio, routine flaring was not permitted and venting had to be limited to 5\% or less (with some venting allowed to power pneumatic devices). Even with low gas to oil ratios, AER reserved the right to intervene even if the 900 cubic meter threshold was not reached.} In practice, however, excess venting was rarely punished. One reason for this is that venting was not actually measured by the government but was only self reported by firms. Estimates from survey research teams find that actual venting was typically more than 10 times the level of reported venting \citep{bib:zavala-araiza-alberta, bib:johnson_comparisons_2017}. Second, the number of enforcement actions AER undertook appears to be quite low relative to the likely number of violations: For example, in our sample of \input{Figures/single_numbers/count_total_sample.tex}sites (and \input{Figures/single_numbers/total_operators_baseline.tex}operators), we observe \input{Figures/single_numbers/sites_exceeds_AERstandard_base.tex}sites (and \input{Figures/single_numbers/opers_exceeds_AERstandard_base.tex}operators) with site-level venting exceeding 604 kg per day. However, during 2015-2019 there were only three events over all of Alberta in which AER undertook enforcement actions against firms due to excess venting.\footnote{In AER enforcement reports, available online at \url{https://www1.aer.ca/compliancedashboard/enforcement.html}, actions regarding excess venting were taken in 2015 against Murphy Oil Company and then in 2019 against Devon Canada Corporation and Fort Calgary Resources Corporation. In at least the first two cases, there were a large number of sites not in compliance and a multi-year history of violations.} Therefore, we view the AER regulation and enforcement as relatively weak, and conclude that any kind of information provision would have informed the firm about lost revenues from methane emissions but likely would not have significantly changed the actual nor perceived probability of regulatory action.\footnote{In addition, the \citet{bib:wang} RCT treatment in particular, described in section \ref{sec:sum_stats}, likely did not change perceived nor actual likelihood of regulatory action because the treatment information did not include any mention of the AER and the \citet{bib:wang} research team did not give site-level nor operator-level emissions testing results to the AER.}
 

Alberta during this time period also had relatively lax restrictions governing required leak detection and repair efforts. Directive 060 required each firm to develop and implement program to find and repair leaks, where the programs were to ``meet or exceed the standards" of CAPP, the Canadian Association of Petroleum Producers. However, this CAPP standard was flexible, not requiring any particular frequency or methodology of leak detection, and stated that ``operators should design a frequency monitoring program best suited for its operations while ensuring maximum cost-effective fugitive emissions reductions" \citep{bib:CAPP}. Therefore, while firms were technically required to do their own leak detection and repair, the regulations were sufficiently vague that firms had significant leeway in their interpretation of the requirements. As a result, it is likely that firms had at least some uncertainty about their methane emissions such that providing information could have a significant effect on beliefs.



\section{Model}
\label{sec:model}

To examine how giving firms information on leakage affects emissions, we present a model where firms are initially uncertain about leakage volumes but can learn leakage volumes either through site visits and testing or through information from an external monitor. For simplicity, we focus on leaks rather than venting. Initially, the firm believes that leakage $L$ for a given site is drawn from a distribution $F$, but without leak detection, does not know the true leakage volume. The firm can choose whether to do leak detection or not ($D =1$ or $D = 0$), where detection informs the firm of the true value of $L$. The firm will also learn $L$ if leak detection is provided by an external monitor. Conditional on learning $L$, the firm can choose how much leakage to repair ($R \in [0, L]$), where the cost of repair level $R$ is given by the function $C_R(R; L)$. We assume that $C_R(0; L) = 0$, $C_R(R; L)$ is weakly increasing on the closed interval $[0, L]$, and that $C_R(R; L)$ is continuous on the half-open interval $(0, L]$, which allows for the possibility that $C_R(R; L)$ includes a fixed cost component.

The firm can also choose whether to do site maintenance or not ($M = 1$ or $M=0$), which reduces the likelihood of future leaks and entails a cost of $C_M$. If the firm does any leak detection, repair, or maintenance, it must make a site visit at cost $C_V$. As production sites are often remote, site visits are costly, which implies a firm would do leak detection, leak repair, and site maintenance all within the same site visit.\footnote{According to \citet{bib:api}, page 121, most leaks are repaired upon detection.} Therefore, the firm's total cost of leak detection, repair, and maintenance given leakage $L$ is:
\begin{equation}
Cost(D, R, M, L) = C_D D + C_R(R; L) + C_M M + C_V \max\{D, 1(R>0), M\}
\label{eq:costs}
\end{equation}

The revenue gains from eliminating leakage depends on leak repair and site maintenance choices as well as leakage amounts and methane market prices. We denote the time-invariant market price of methane as $P$. We assume that exerting repair effort $R \in [0, L]$ leads to leakage from existing leaks to be equal to $L - R$. We also assume that as part of doing a site visit, the firm undertakes routine maintenance and that routine maintenance reduces expected new leakage by $\delta$, where $\delta$ is some weakly positive number. Therefore, the firm's expected future leakage $L_F$ as a function of repair and maintenance choices is given by equation \ref{eq:future_L} below. The firm's revenue from repair and maintenance choices is given by equation \ref{eq:revenues} below. The firm's total payoff is revenues in equation \ref{eq:revenues} minus costs in equation \ref{eq:costs}.
\begin{align}
L_F(L, R, M) & = L - R + \delta (1 - M) \label{eq:future_L} \\
Rev(L, R, M) & = P R + P \delta M \label{eq:revenues}
\end{align}

We make some simplifying assumptions that reduce the number of cases that must be examined while focusing on the interesting cases. First, we assume $C_M = 0$: Assuming a negligible cost of site maintenance implies that conditional on doing a site visit, the firm will always do site maintenance. Second, we assume that $P \delta < C_V$, meaning the expected value of future leaks is not sufficiently high on its own to incentivize the firm to make a site visit. Therefore, the firm will require either a sufficiently high expected leakage (in the case of not receiving external monitoring) or sufficiently high actual leakage (in the case of receiving external monitoring) to justify a site visit. These two assumptions together also imply that $M = \max\{D, 1(R>0)\}$ -- the firm only does site maintenance (and similarly only does a site visit) if the firm finds it worthwhile to do leak detection or leak repair.


We solve the model first for the case where the firm is not informed of $L$ by an external monitor, and solve by backward induction: Conditional on doing a site visit and measuring leakage ($D=1$) to learn the leakage volume $L$ and therefore also do maintenance, the firm will do choose leakage repair effort $R$ that maximizes $Rev(L, R, M=1) - Cost(D=1, R, M=1, L) = PR - C_R(R; L)$. We write the firm's profit-maximizing repair effort as $R^{*0}(L)$ and show in the web appendix that $R^{*0}(L)$ is weakly increasing in $L$.\footnote{Specifically, we define $R^{*0}(L, P) = \max \{x \in [0, L] | Px - C_R(R; L) = \max_{y \in [0, L]} (Py - C_R(y; L)) \}$. In other words, if the firm is indifferent between multiple repair effort levels, we assume it will choose the one that maximizes leakage repair.} Then, given this repair policy function, the firm will only do a site visit to detect leaks if $\tilde{V} = 1$, where $\tilde{V}$ comes from integrating over all the range of possible leakage values:
\begin{equation}
\tilde{V} = 1(E [\max\{ P R^{*0}(L) - C_R(R^{*0}(L), L), 0 \} ] + P \delta - C_D - C_V \geq 0)
\label{eq:v_tilde}
\end{equation} 

The other case is where the firm is informed of $L$ by an external monitor: The  firm then chooses a repair effort $R$ which maximizes $Rev(L, R, M) - Cost(D, R, M, L) = P R  - C_R(R; L) + (P \delta - C_V) \cdot 1(R >0)$, where now the choice to conduct any positive amount of repair will therefore incur a site visit cost $C_V$ and a maintenance benefit $P\delta$. We denote the optimal repair choice in this setting as $R^{*1}(L)$. $R^{*0}(L)$ and $R^{*1}(L)$ solve very similar objective functions where the only difference is that for $R^{*1}(L)$, the fixed cost of any positive repair is increased by $C_V - P \delta > 0$. The assumptions on $C(R; L)$ then imply the following, with proofs in the web appendix:
\begin{align}
R^{*1}(L) & \leq R^{*0}(L) \label{eq:R_star_inequality} \\ 
R^{*1}(L) > 0 & \Rightarrow R^{*1}(L) = R^{*0}(L) > 0 \label{eq:R_stars_equal_if_R1_positive} \\ 
R^{*0}(L) = 0 & \Rightarrow R^{*1}(L) = R^{*0}(L) = 0  \label{eq:R1_is_zero_if_R0_is_zero}
\end{align}

The interpretation of expression \ref{eq:R_star_inequality} is that conditional on knowing leakage, learning about leakage from an external monitor leads to weakly less repair effort than if the operator learned about leakage from its own site visit. Expression \ref{eq:R_stars_equal_if_R1_positive} means that if a firm would at least partially repair a leak after being informed by an external monitor, a firm would also exert the same repair effort if it learned about the leak through its own site visit. Expression \ref{eq:R1_is_zero_if_R0_is_zero} means that if a firm would do zero leak repair after learning about it through its own site visit and leak detection, it would do zero leak repair if it learned of the leak from an external monitor.

Expressions \ref{eq:R_star_inequality} through \ref{eq:R1_is_zero_if_R0_is_zero} imply we can segment the range of potential leakage $L$ into three bins: $\mathbb{L}_1$, $\mathbb{L}_2$, and $\mathbb{L}_3$.\footnote{Whether or not each of the leakage bins is non-empty depends on the cost function. For example, if $C_R(R; L) = \alpha R$, where $\alpha > P$, then both $\mathbb{L}_2$ and $\mathbb{L}_3$ are empty and the firm never does leak repair. If $C_R(R; L) = 0$, then $\mathbb{L}_1$ is empty.} In the lowest leakage bin $\mathbb{L}_1$, $R^{*1}(L) = R^{*0}(L) = 0$. These are small leaks that when they are known about, they are not repaired -- regardless of whether the firm learns about them through its own site visit or through an external monitor. In the middle leakage bin $\mathbb{L}_2$, $R^{*1}(L) = 0 <R^{*0}(L)$. These are leakage levels at which the firm would not repair the leak if it is informed by the external monitor but would repair (at least to some extent) if it detected them on its own. The reason for the difference is that if the firm receives information from the external monitor, its fixed costs of repair increase by $C_V - P \delta$, which disincentivizes repair. The third leakage bin $\mathbb{L}_3$ is high leakage which, when known about by the firm, gets equal positive repair effort regardless of whether the firm learned about it through its own leak detection efforts or from an external monitor ($R^{*1}(L) = R^{*0}(L) > 0$). 

The treatment effect of providing information is therefore theoretically ambiguous, depending on both actual leakage levels and $\tilde{V}$ -- what the firm would do in the absence of receiving information from the external monitor. We summarize the effects in table \ref{tab:model_treatment_effects}. The table shows that if $\tilde{V}=0$ (the firm would not do a site visit if it does not receive leakage information from an external monitor), providing information reduces leakage as long as leakage is sufficiently high ($L \in \mathbb{L}_2 \cup \mathbb{L}_3$), as providing information leads the firm to make a site visit and therefore repair leaks and perform site maintenance. However, if $\tilde{V} = 1$, providing information leads to an increase in leakage when leakage is sufficiently low ($L \in \mathbb{L}_1 \cup \mathbb{L}_2$). This adverse effect comes from information disincentivizing the firm from making a site visit, which increases emissions both through disincentivizing repair (current leaks are not fixed) as well as through disincentivizing maintenance (new leaks are more likely to appear in the future). 

\begin{table}[t]
\begin{tabular}{ l c c c c }
\hline
   & \multicolumn{4}{c}{Future leakage $L_F$} \\
\cline{2-5}
   & No external info & External info & Treatment Effect & Sign  \\
\hline
$\tilde{V}=0$ \\
 $\; \; \; \;  L \in  \mathbb{L}_1$   & $L + \delta$ & $L + \delta$ & $0$ & 0  \\
 $\; \; \; \; L \in  \mathbb{L}_2$  & $L + \delta$ & $L - R^{*1}(L)$ & $- R^{*1}(L) - \delta$ & - \\
 $\; \; \; \; L \in  \mathbb{L}_3$  & $L + \delta$ & $L - R^{*1}(L)$ & $- R^{*1}(L) - \delta$  & - \\
\hline
$\tilde{V}=1$ \\ 
 $\; \; \; \; L \in  \mathbb{L}_1$  & $L$ & $L+\delta$ & $\delta$ & + \\
 $\; \; \; \; L \in  \mathbb{L}_2$  & $L - R^{*0}(L)$ & $L + \delta$ & $R^{*0}(L) + \delta$ & + \\
 $\; \; \; \; L \in  \mathbb{L}_3$  & $L - R^{*0}(L)$ & $L - R^{*1}(L)$ & 0 & 0 \\
\hline
\end{tabular}
\caption{A summary of the firm's future leakage $L_F$ as a function of $\tilde{V}$ and leakage level $L$, where $\tilde{V}$ is the firm's site visit choice in the case that it does not receive information on leakage from the external monitor. The treatment effect of providing leakage information to the firm and is the difference between the previous two columns.}
\label{tab:model_treatment_effects}
\end{table}


 
\section{Data description}
\label{sec:sum_stats}


To examine the effect of information, we use data from \citet{bib:wang} which conducts and analyzes an RCT examining the effects of methane leak and venting monitoring at oil and gas production sites in Alberta in 2018 and 2019. There are a total of \input{Figures/single_numbers/count_total_sample.tex}sites in the data, with about \input{Figures/single_numbers/percent_treatment.tex}\unskip\% of the sites assigned to treatment.\footnote{One of the original goals of the \citet{bib:wang} study was to examine the effect of monitoring frequency, and so there were three intensities of treatment: Annual, biannual, and triannual visits. While annual treatment sites were only surveyed at baseline and endline, biannual treatment sites were also surveyed approximately six months after baseline (March 2019), and triannual treatment sites were surveyed both approximately four and approximately eight months after baseline (November 2018 and May 2019). For treatment sites, operators were informed of emissions after each site survey, meaning that operators at triannual sites received more frequent monitoring information than operators at biannual sites who in turn received more frequent monitoring information than operators at annual treatment sites. Because the midline surveys in November 2018, March 2019, and May 2019 were not done for control and annual treatment groups, we focus our analysis on the measured emissions volumes in the baseline and endline surveys. For power reasons, our analysis in this paper focuses on comparing treatment sites to control sites and not on examining the effect of monitoring frequency.}  Randomization was done at the site level. For all sites, the research survey team measured emissions (both venting and leakage) at the component level both at baseline (between August and October 2018) and endline (between August and October 2019). Leakage and venting methane emissions were measured using optical gas imagine (OGI).
% OGI..., a technology that can detect emissions as low as ... and that has an accuracy of ... (add citation). . 

For treatment sites, operators were informed about emissions in two ways: First, at the time of the survey, the research survey team physically tagged leaking components where physically possible, allowing operators who do site visits to quickly identify leaking components. Second, following each survey, the research survey team sent a report to the site operator giving detailed emissions information on all venting and leakage found via spreadsheets, photos, videos, and a slide deck. In contrast, the research survey team did not inform operators of control sites about their measured emissions, neither through tagging nor through reports, and control site operators were not even informed that site methane emissions monitoring was being done. This was a moderately costly study to implement as typical OGI with methane emissions quantification costs around \$800-\$1,000 per site visit and additional costs came from compiling reports for site operators at treatment sites.\footnote{The above are on the higher side reflecting that OGI with quantification of emissions using optimal imaging software takes about twice as long as OGI without emissions quantification. \citet{bib:EDF_response_to_API} reports estimated costs of OGI between \$250 and \$800 per site.} 



We caveat some drawbacks of the \citet{bib:wang} RCT. First, the RCT was not preregistered as preregistration is still quite rare in engineering. Second, the RCT was not designed to identify the heterogeneous treatment effects of information as a function of baseline emissions. For example, there was no power calculation done by baseline emission status nor stratification of treatment by baseline emission status.\footnote{Web appendix \ref{sec:gelman_carlin} includes a discussion of power as well as type-M and type-S errors following \citet{bib:gelman_carlin}.} Third, randomization was done at the site level rather than at the operator level. While there is little we can do to address these first two concerns, we address the third by clustering standard errors by operator and performing a test for SUTVA violations. Our SUTVA test, discussed in web appendix section \ref{sec:SUTVA}, does not find significant evidence of spillovers. 


In table \ref{tab:sumstats}, we display summary statistics, and find significant heterogeneity both in emissions and production. Mean baseline leakage is \input{Figures/single_numbers/leakage_mean_at_baseline.tex}kg/day and the distribution has both a long right tail and a mass point at zero: The ratio of the 95th percentile of leakage to the median leakage is \input{Figures/single_numbers/leakage_ratio_p95_p50_baseline.tex}and \input{Figures/single_numbers/frac_zero_baseline_leakage.tex}\unskip\% of the sites have zero leakage at baseline. Consistent with data from other studies, venting is significantly larger than leakage, with mean venting equal to \input{Figures/single_numbers/venting_mean_at_baseline.tex}kg/day. Mean oil production at baseline is \input{Figures/single_numbers/oil_mean_at_baseline.tex}cubic meters per month; mean natural gas production at baseline is \input{Figures/single_numbers/gas_mean_at_baseline.tex}thousands of cubic meters per month. While there is significant variation in whether sites predominately produce natural gas or oil, \input{Figures/single_numbers/perc_positive_nat_gas.tex}\unskip\% of our sites report producing at least some natural gas, suggesting that almost all sites (if not all) are connected to natural gas pipelines and therefore that leak repair can increase revenues.\footnote{\input{Figures/single_numbers/count_missing_gas_at_baseline.tex}of the \input{Figures/single_numbers/count_total_sample.tex}sites in the data have missing gas and oil production data.}

\begin{table}[t!]	
\begin{tabular}{l c c c c c}
\hline
\input{Figures/summary_stats_outliers.tex}
\hline
\end{tabular}

\caption{Tables of summary statistics for baseline and endline, including leakage, venting, and oil and gas production. Emissions are measured in kilograms per day, natural gas is measured in thousands of cubic meters per month, and oil production is measured in cubic meters per month.}
\label{tab:sumstats}
\end{table}

\begin{table}[hp!]
\begin{tabular}{l c c c }
Panel (a): Full sample \\
\hline \hline 
\input{Figures/balance_table_TConly.tex}
\hline \hline \\
Panel (b): Zero baseline leakage sites \\
\hline \hline
\input{Figures/balance_table_zero_leakage.tex}
\hline \hline
\end{tabular}
\caption{Balance test table. Panel (a) is balance for the full sample; panel (b) is balance for the subset of the sample with zero baseline leakage. The first column gives the mean of variables, measured at baseline, for the control group, with the standard deviation below in parantheses. The second column gives the mean of variables, measured at baseline, for cases where the site is in any of the three treatment groups, with the standard deviation below in parantheses. The third column is the difference between the first two columns. The standard error is below in parentheses and is clustered by operator.}
\label{tab:balance}
\end{table}


We find that for the most part, treatment and control are well balanced. Table \ref{tab:balance} is a balance table of baseline characteristics, where panel (a) displays balance tests for the full sample and panel (b) displays balance tests for the subset of the sample with zero measured baseline leakage. For all variables measured at baseline -- site establishment date, venting, leakage, oil production, and gas production -- we are unable to reject at the 5\% level that the the mean in the treatment sites differs from the mean in the control sites.\footnote{The median site establishment date is \input{Figures/single_numbers/median_site_start.tex}\unskip. The oldest site in our data was established in \input{Figures/single_numbers/min_site_start.tex}\unskip; the youngest site was established in \input{Figures/single_numbers/max_site_start.tex}\unskip.} The one variable which is close to showing a difference is baseline venting, and that only for the sites with zero baseline leakage (panel b): There, a test that treatment and control groups have the same mean venting yields a p-value of \input{Figures/single_numbers/test_vent_zero_base_leakage.tex}\unskip. Given the concern of imbalance in this setting, we therefore include controls for baseline venting in some of the regression specifications in section \ref{sec:zero_baseline_leakage}. In addition, web appendix section \ref{sec:zero_baseline_leakage_appendix} includes additional specifications where we interact treatment with baseline venting.

\section{Estimating treatment effects}
\label{sec:treat_effects}

To analyze the effects of treatment, we focus on two outcome variables: Leakage and total emissions, where the latter is the sum of leakage and venting. We first discuss the overall distribution of treatment effects using all observations in our data. Then in section \ref{sec:zero_baseline_leakage}, we perform additional robustness checks for sites with zero baseline leakage.

\subsection{Full sample}
\label{sec:treat_effects_full_sample}


\begin{figure}[t!]
    \centering
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/lpoly_log1_leakage_on_log1_leakage.pdf} 
        \caption{Predicted leakage conditional on baseline leakage by treatment status.} \label{fig:lpoly_log1_leakage_on_log1_leakage}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/lpoly_log1_leakage_on_log1_leakage_DIF.pdf} 
        \caption{Difference between treatment and control from panel (a).} \label{fig:lpoly_log1_leakage_on_log1_leakage_DIF}
    \end{subfigure}

    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/lpoly_log1_emissions_on_log1_leakage.pdf} 
        \caption{Predicted emissions conditional on baseline leakage by treatment status.} \label{fig:lpoly_log1_emissions_on_log1_leakage}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/lpoly_log1_emissions_on_log1_leakage_DIF.pdf} 
        \caption{Difference between treatment and control from panel (c).} \label{fig:lpoly_log1_emissions_on_log1_leakage_DIF}
    \end{subfigure}
    
\caption{Panel (a) shows local polynomial regression of log(1 + endline leakage) on log(1 + baseline leakage), separate for treatment and control groups. Panel (b) shows the difference between predicted log(1 + endline leakage) for treatment and predicted log(1 + endline leakage) conditional on control. Panel (c) shows local polynomial regression of log(1 + endline emissions) on log(1 + baseline leakage), separate for treatment and control groups. Panel (d) shows the difference between predicted log(1 + endline emissions) conditional on treatment and predicted log(1 + endline emissions) conditional on control. 95\% confidence intervals in panels (b) and (d) are calculated using \protect\input{Figures/single_numbers/bs_rep_count_lpoly.tex}bootstrap repetitions clustered at the operator level.}
    \label{fig:lpoly_log1_leakage_emissions}
\end{figure}


To get an overall sense of the treatment effect over all observations, we use a nonparametric specification where we project the log of one plus endline leakage on the log of one plus baseline leakage, doing so separately for treatment and control groups.\footnote{We use a local polynomial regression with a degree of zero and a bandwidth of one.} The results are plotted in panel (a) of figure \ref{fig:lpoly_log1_leakage_emissions}. There, we find that treatment at low baseline leakage sites is associated with an increase in endline emissions whereas treatment at high baseline leakage sites is associated with a decrease in endline emissions. We plot the difference between the treatment and control group predictions in panel (b) along with 95\% confidence intervals derived from \input{Figures/single_numbers/bs_rep_count_lpoly.tex}bootstrap repetitions (clustered at the operator level).

We plot similar graphs in panel (c) of figure \ref{fig:lpoly_log1_leakage_emissions} where we estimate predicted log of one plus total endline emissions (leakage plus venting) as a function of treatment and the log of one plus baseline leakage. We find similar results: Treatment appears to decrease total emissions when baseline leakage is high but increase total emissions when baseline leakage is low, although the latter does not appear to be statistically significant (see panel (d)). 

These non-parametric regressions show evidence that when site operators receive the good news that leakage is low, endline leakage increases. These results are consistent with the idea that good news of low leakage crowds out site visit and maintenance activity, reducing repair for these low-volume leaks and increasing the likelihood of future leaks.\footnote{Unfortunately, our data includes no direct observation of site visit and maintenance activity. While our data does include some leak repair activity, it is limited to only a subset of treatment site leaks, and never observed for control site leaks. Further details are in web appendix section \ref{sec:repairs}.} In the next section, we conduct additional robustness exercises for the subset of sites with zero measured leakage at baseline.


\subsection{Sites with zero baseline leakage}
\label{sec:zero_baseline_leakage}



To further examine treatment effects for sites with zero baseline leakage, we first graphically compare the CDF of outcome variables for the treatment group with that of the control group. Our results for endline leakage are in panel (a) of figure \ref{fig:endline_cdf_base_leak_is_zero} where we find that the treatment group CDF is always weakly to the right of the CDF of the control group, and is strictly so outside of the mass points at zero. This implies that treatment with information causes the site to have higher leakage at endline relative to control sites. We also find similar results for total emissions in panel (b) of figure \ref{fig:endline_cdf_base_leak_is_zero}, where the CDF of endline emissions for the treatment group is weakly to the right of the CDF of endline emissions for the control group. Similar results hold when we examine the change in emissions over time, which accounts for the fact that treatment sites have somewhat higher baseline venting than control sites (figure \ref{fig:cdf_change_emissions_base_leak_is_zero} in web appendix section \ref{sec:zero_baseline_leakage_appendix}). 

\begin{figure}[t!]
    \centering
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_endline_leakage_base_leak_is_zero.pdf} 
        \caption{Leakage} \label{fig:cdf_endline_leakage_base_leak_is_zero}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_endline_emissions_base_leak_is_zero.pdf} 
        \caption{Total Emissions} \label{fig:cdf_endline_emissions_base_leak_is_zero}
    \end{subfigure}
    \caption{The cumulative distribution functions (CDFs) of leakage and total emissions at endline, conditional on zero leakage at baseline, comparing treatment and control groups. Panel (a) shows the CDFs of endline leakage. Panel (b) shows the CDFs of total endline emissions, summing over leakage and venting.}
    \label{fig:endline_cdf_base_leak_is_zero}
\end{figure}




Next, we use an OLS specification where we allow for additional controls. Our results are in panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}. Columns 1 and 2 examine the effect of treatment on endline leakage; columns 3 and 4 examine the effect of treatment on endline total emissions (leakage plus venting). Columns 2 and 4 include added controls for baseline oil and gas production, as well as levels of  baseline venting.\footnote{Specifically, we control for the log of one plus baseline oil production, log of one plus natural gas production, and log of one plus baseline venting.} Our estimates in columns 1 and 2 show that at sites with zero baseline leakage, treatment increased endline leakage by approximately \input{Figures/single_numbers/treat_effect_leak_no_controls_zero_baseline_leakage_OLS.tex}kg of methane per day (without controls) and \input{Figures/single_numbers/treat_effect_leak_w_controls_zero_baseline_leakage_OLS.tex}kg of methane per day (with controls), with the estimates in both regressions marginally significant. Our estimates in columns 3 and 4 show that treatment causes a statistically significant increase in total emissions of \input{Figures/single_numbers/treat_effect_emiss_no_controls_zero_baseline_leakage_OLS.tex}kg methane per day (without controls) or \input{Figures/single_numbers/treat_effect_emiss_w_controls_zero_baseline_leakage_OLS.tex}kg methane per day (with controls).


Third, we use the two-part model of \citet{bib:belotti}, modeling the outcome variable $y_i$ for site $i$ as depending on baseline characteristics $X_i$ and treatment status $T_i$, where we jointly estimate the probability that $y_i$ is positive (equation \ref{eq:twopm_part_1}) and the expectation of the log of $y_i$ conditional on $y_i$ being positive (equation \ref{eq:twopm_part_2}). The advantages of \cites{bib:belotti} approach is that the log transformation reduces the influence of outlier observations of $y_i$ while also allowing for zeros and being robust to the unit of measurement for $y_i$ \citep{bib:chen_roth, bib:mullahy_norton}. The major drawback is that it requires estimating more parameters than a single index model and therefore reduces precision of the estimated treatment effect.
\begin{align}
p(y_i > 0) &  = \phi(X_{i}, T_i) \label{eq:twopm_part_1} \\
E( \log(y_i) | y_i > 0) &  = \psi(X_{i}, T_i) \label{eq:twopm_part_2}
\end{align}

\begin{table}[t!]
\begin{tabular}{l*{4}{c}} 
Panel (a): OLS:  \\
 \hline\hline
\input{Figures/reg_zero_baseline_leakage_OLS_fragment.tex}
\hline \hline \\ 
Panel (b): Two-part model:  \\
\hline \hline
\input{Figures/reg_zero_base_leakage_TWOPM_fragment.tex}
\hline \hline
\end{tabular}
\caption{Estimates of treatment effects using OLS (panel a) and the two-part model of \citet{bib:belotti} (panel b), conditional on zero baseline leakage. Columns 1 and 2 examine the effect of treatment on endline leakage; columns 3 and 4 examine the effect of treatment on endline total emissions. Standard errors are in parentheses and are clustered by operator. Controls included in columns 2 and 4 are the natural log of one plus baseline venting (in kilograms per day), the natural log of one plus baseline oil production (in cubic meters per day), and the natural log of one plus baseline natural gas production (in thousands of cubic meters per day). The full regression results are in web appendix section \protect\ref{sec:zero_baseline_leakage_appendix}.}

\label{tab:reg_zero_baseline_leakage_OLS_TWOPM}
\end{table}


Equation \ref{eq:twopm_part_1} is estimated using a logit specification and equation \ref{eq:twopm_part_2} is estimated using a general linear model. Computing the predicted outcome and therefore the treatment effect involves multiplying the predicted probability of positive $y$ (from equation \ref{eq:twopm_part_1}) with the expected $y$ conditional on $y$ being positive (from equation \ref{eq:twopm_part_2}). To integrate over the residuals in the general linear model, we use the smearing technique of \citet{bib:duan}. Our computed average treatment effect is the average of all of the site-level predicted treatment effects.

Our two-part model results are in panel (b) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}. In columns 1 and 2, we examine the effect of treatment on leakage. Column 1 includes no control variables and shows that treatment increases emissions by \input{Figures/single_numbers/treat_effect_leak_no_controls_zero_baseline_leakage_TWOPM.tex}kilograms per day, identical to the corresponding OLS results (panel a, column 1). In column 2, we include controls for baseline venting, oil production, and natural gas production, and find a slightly higher treatment effect (\input{Figures/single_numbers/treat_effect_leak_w_controls_zero_baseline_leakage_TWOPM.tex}kilograms per day) than our OLS estimates in panel (a) column 2. For both columns 1 and 2, our bootstrapped 95\% confidence intervals are wide but do allow us to reject a zero average treatment effect. 

Then in columns 3 and 4 of panel (b) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}, we examine the effect of treatment on total emissions. In column 3 (no controls), we estimate that treatment increases total emissions by \input{Figures/single_numbers/treat_effect_emiss_no_controls_zero_baseline_leakage_TWOPM.tex}kilograms per day, identical to our OLS results in panel (a) column 3. In column 4, we include controls for baseline venting, oil production, and natural gas production.\footnote{Because there are very few observations with zero total emissions at endline, these controls are only included in the GLM specification (equation \ref{eq:twopm_part_2}) but not in the logit specification (equation \ref{eq:twopm_part_1}).}  There, we find an estimated treatment effect of \input{Figures/single_numbers/treat_effect_emiss_w_controls_zero_baseline_leakage_TWOPM.tex} kilograms per day, somewhat larger than our OLS estimate of \input{Figures/single_numbers/treat_effect_emiss_w_controls_zero_baseline_leakage_OLS.tex}kilograms per day from panel (a) column 4. Again, our confidence intervals are wide, and only in column 3 with no controls (but not in column 4 with controls) do our bootstrapped confidence intervals for the average treatment effect allow us to reject a zero average treatment effect.

Overall, our results in table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM} show largely consistent results: Information treatment at sites with zero baseline leakage leads to an increase of about \input{Figures/single_numbers/median_leak_effect_round.tex}kg/day of leakage and \input{Figures/single_numbers/median_emiss_effect_round.tex}kg/day of total emissions. This translates to approximately \$\input{Figures/single_numbers/median_leak_effect_DOLLARS.tex}\unskip/day of damage from leakage and \$\input{Figures/single_numbers/median_emiss_effect_DOLLARS.tex}\unskip/day of damage from emissions (leakage plus venting) for each treated site. We include additional empirical results in web appendix section \ref{sec:empirical_appendix}.

Our results are consistent with the idea in the model that at least some site operators with zero measured leakage at baseline believed that it was worthwhile to do a site visit (including leak detection and repair as needed)  -- and therefore, in the notation of the model, that $\tilde{V} = 1$. Therefore, providing external monitoring of leakage seems to have crowded out their site visit, leak detection, and maintenance effort, which in turn increased endline emissions. 


\section{Discussion}
\label{sec:policy_implications}

We next discuss policy implications. We first discuss whether alternative information provision can perform better. Then we discuss general opportunities and challenges for policy improvement given emissions uncertainties and evolving technology.  



\subsection{Can conditional information provision improve outcomes?}
\label{sec:optimal_threshold}

Given that providing firms information on baseline leakage increases endline leakage and total emissions when baseline leakage is low, can a policy maker improve outcomes by only giving information on leakage when leakage exceeds a certain threshold? To answer this, we build on the model in section \ref{sec:model} and examine what happens when the monitor gives information to the firm only when $L \geq H$, where $H$ is some threshold chosen by the monitor. This is a generalization of the control and treatment rules in the experiment, with control group corresponding to $H=\infty$ (no information ever given) and treatment group corresponding to $H=0$ (information always given). To illustrate these effects, we focus on the setting where $C_R(R; L) = 1(R>0) \cdot C_R$ with $C_R \geq 0$, meaning that leakage repair has weakly positive fixed costs but zero variable costs, which implies that either $R = 0$ (leak is not repaired at all) or $R = L$ (leak is completely repaired).


\begin{figure}[t!]
    \centering
    \begin{subfigure}[t]{0.45\textwidth}
        \centering
        \includegraphics[scale=0.4, page=15, clip=true, trim={1.2cm 0cm 11cm 4cm}]{Figures/Powerpoint_Graphics/graphic_info_threshold_depends_on_L.pdf} 
        \caption{Case where if no information is ever given ($H=\infty$), the firm does not make a site visit ($\tilde{V} = 0$).} 
        \label{fig:theory_threshold_no_visit_in_control}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.45\textwidth}
        \centering
        \includegraphics[scale=0.4, page=16, clip=true, trim={1.2cm 0cm 11cm 4cm}]{Figures/Powerpoint_Graphics/graphic_info_threshold_depends_on_L.pdf} 
        \caption{Case where if no information is ever given ($H=\infty$), the firm makes a site visit ($\tilde{V} = 1$).} 
        \label{fig:theory_threshold_does_visit_in_control}
    \end{subfigure}
    \caption{Operator visit (including maintenance), detection, and repair decisions as a function of leakage $L$ and the information threshold $H$. For both panels, the lower right triangle $L < H$ corresponds to the settings where no information on leakage is given to the firm; the upper left triangle $L \geq H$ corresponds to the setting where information on leakage is given to the firm. The bold red line in each panel demarcates the scenarios in which leak repair is made}
    \label{fig:theory_threshold_firm_decision}
\end{figure}


We illustrate the firm's actions as a function of $L$ and $H$ in the two panels in figure \ref{fig:theory_threshold_firm_decision}. The lower right triangle $L < H$ in each panel corresponds to the setting where no information on leakage is given to the firm; the upper left triangle $L \geq H$ in each panel corresponds to the setting where information on leakage is given to the firm. 

Recall that the treatment effect depends on $\tilde{V}$ which is the firm's site visit decision if $H=\infty$. In figure \ref{fig:theory_threshold_firm_decision} panel (a), we illustrate the situation where if $H=\infty$, the firm would not do a site visit ($\tilde{V} = 0$). In this case, lowering the threshold $H$ increases the probability of site visit and repair up until the point that $H = -\delta + (C_V + C_R)/P$. For this setting, setting any $H \in [0, - \delta + (C_V + C_R)/P ]$ maximizes the probability that leaks are repaired.

The other situation is where if $H=\infty$, the firm would do a site visit ($\tilde{V} = 1$). We illustrate this case in figure \ref{fig:theory_threshold_firm_decision} panel (b). We define a threshold value $B$ such that if the firm knows that the information threshold is $H=B$ and the firm receives no information about leaks, the firm is indifferent between doing a site visit and not doing a site visit; this threshold $B$ is implicitly defined in equation \ref{eq:define_B}: 

\begin{equation}
E[\max\{P L - C_R, 0\} | L < B] + P \delta - C_V - C_D = 0
\label{eq:define_B}
\end{equation}

If $H \in (0, B)$, then not receiving information from the monitor will disincentivize the firm from making a site visit relative to $H > B$. For the subset of those cases with $H \in (-\delta + (C_V + C_R)/P, B)$, the probability of doing a site visit and making a repair is even lower than when $H=0$ (the treatment case). This is illustrated by the center part of the bold red line in figure \ref{fig:theory_threshold_firm_decision} panel (b) which shows an especially high probability of no repair for intermediate values of $H$. In other words, if the firm plans to visit the site when $H=\infty$, then intermediate values of $H$ can lead to worse repair and emissions outcomes  than both the $H = 0$ (always informing the firm) and the $H = \infty$ (never informing the firm) cases.

While an intermediate threshold never strictly dominates both the $H = 0$ and $H = \infty$ cases for a single site, an intermediate threshold can strictly dominate when aggregating over heterogeneous sites. For example, in figure \ref{fig:theory_threshold_firm_decision}, the height of the bolded red lines is one-to-one with the probability of no repair, and if we assume that $\delta = 0$ (no future leaks), a social planner's goal is to minimize the probability of no repair. Assuming that for most sites, the operator would make a visit in the absence of information ($\tilde{V}=1$ as in panel (b) of figure \ref{fig:theory_threshold_firm_decision}) but that there is at least one site for which the operator would not make a visit in the absence of information ($\tilde{V}=0$ as in panel (a) of figure \ref{fig:theory_threshold_firm_decision}), then setting a threshold $H=B$ could lead to a higher average rate of leak repair than both always informing ($H=0$) and never informing ($H=\infty$).\footnote{This argument also assumes that the costs $C_V$, $C_R$, and $C_D$ are all identical across all sites and that for each site, the term on the left hand side of equation \ref{eq:v_tilde} is arbitrarily close to zero, such that in the case of $H = \infty$, each site operator is nearly indifferent between visiting and not visiting.} However, in general for heterogeneous sites, there is no guarantee that there exists an intermediate threshold that dominates both the $H = 0$ and $H = \infty$ cases, and even if one exists, finding that threshold is very challenging because it depends on the firm's ex ante beliefs and costs, all of which a regulator typically has very little information on. 


\subsection{Monitoring, efficiency, and technical constraints}
\label{sec:first_best}


While the uncertainty reduction effects of methane monitoring is of limited value to reducing emissions on its own, methane monitoring has a second important value: Providing a measurable quantity for which to base methane regulation. For example, both Pigouvian methane taxes and methane emission credits require credible measures of methane emissions. When firms have correct information about methane emissions \emph{and} face the true social cost of methane (via taxes or credits), firms will choose the socially optimal level of repair and emissions, achieving the first-best outcome. Methane monitoring therefore serves to transform a non-point-source pollution problem into a point-source pollution problem where point-source solutions can be implemented.


However, transforming methane emissions to a point-source problem requires emissions monitoring that is high frequency, geographically precise, with a low minimum detection threshold, and sufficiently low cost: High frequency monitoring is important to ensure that all emissions events are appropriately detected and taxed, and especially important for early identification of right-tail emission events that should be repaired quickly. Geographic precision is important for attributing emissions to source and therefore imposing the true social cost of methane on the responsible operator.  Low minimum detection thresholds are important to capture all methane emissions and not just the right tails. And of course, low cost is important to ensure takeup. In the absence of monitoring that satisfies these conditions, methane emissions from oil and gas production remains a nonpoint source emissions problem and therefore cannot be regulated using point source pollution regulation such as Pigouvian taxes. Furthermore, the absence of technology that satisfies these four conditions will often mean that firms are also uncertain about their own emissions, which uncertainty adds an additional barrier to achieving first-best outcomes.

Unfortunately, most current technology is not yet able to monitor emissions technology in a way that is high frequency, geographically precise, with a low minimum detection threshold, and at low cost. For example, site-level emissions detection technologies such as optical gas imaging (OGI) has low detection thresholds -- typically less than 0.2 kg/hour \citep{bib:ravikumar, bib:zimmerle} -- and can measure emissions at the component level. However, OGI requires costly site visits and so cannot be used with high frequency. Aircraft- and drone-mounted monitoring, such as that using LiDAR, is able to monitor hundreds of sites per day. However, the minimum detection threshold (3 kg/hour according to one firm) is significantly higher than that of OGI and can identify emissions at equipment level (such as tank or wellhead) but not at component level.\footnote{\url{www.bridgerphotonics.com}.} Satellite detection of methane is continuing to improve: The new MethaneSAT methane satellite recently launched by EDF will be able to monitor global emissions every 95 minutes, with data publicly disseminated days later. However, this satellite will have both a high minimum detection threshold and low geographic precision, with a minimum detection threshold of 5-80 kg/hour at the 1 square kilometer level and a minimum detection threshold of 500 kg/hour at the site-level \citep{bib:methanesat}. One more promising approach is low minimum-threshold site-level monitoring via stationary towers that autonomously monitors emissions via sensors and then wirelessly transmits data on emissions to operators.\footnote{\url{www.longpathtech.com}} However, this technology is new, has large installation costs, and therefore has had limited take up as of yet. 


Recent work has explored how to achieve first- or second-best emissions outcomes in spite of the limitations of current emission monitoring technology. For example, \citet{bib:dunklewerner_qiu} model a setting where firms choose their own methane abatement effort and examine the extent to which auditing sites and fining sites with high emissions can reduce methane emissions. They show that if audits and fines are far more effective at reducing methane if fines are large and if the regulator uses well observables and remote sensing data to target audits. \citet{bib:cicala} explore a setting in which a regulator can observe aggregate emissions (e.g., through remote sensing) but not individual firm-level emissions. It imposes a policy where each firm pays taxes on its actual emissions if it reports its emissions to the regulator. If it does not report its emissions to the regulator, it pays taxes based on average emissions of all nonreporting firms. Such a policy leads to a first-best outcome via an ``unraveling'' effect where all firms choose to disclose their actual emissions. Both the \citet{bib:dunklewerner_qiu} and \citet{bib:cicala} models assume that firms know their own true propensity to emit. However, our empirical work showing a non-zero treatment effect of information implies that firms are uncertain about their own true emissions propensity. Such uncertainty therefore is an additional barrier to achieving the first-best outcome.




The fact that firms are uncertain about their own emissions levels has also affected current policy design. For example, in the United States, the New Source Performance Standards (NSPS) therefore includes requirements for regular leak detection and repair at oil and gas production sites both as a way to reduce emissions and reduce emissions uncertainty.\footnote{Just as detecting firm-level emissions is challenging, measuring compliance with leak detection and repair requirements is also challenging, and therefore NSPS standards are effectively unenforceable. This is acknowledged in \citet{bib:EPA_NSPS_2024} which says that that such standards are used in settings in which ``it is not feasible to prescribe or enforce a standard of performance” and that such settings include those ``when the pollutant cannot be emitted through a conveyance designed to emit or capture the pollutant, or when there is no practicable measurement methodology for the particular class of sources."} Recent NSPS updates also work to reduce emissions uncertainty through a new super-emitter program in which remote sensing of high emissions is communicated to the EPA and then in turn to the operator \citep{bib:EPA_NSPS_2024}. Finally, the recent Inflation Reduction Act has included not only a net methane fee for large emitters as part of the Greenhouse Gas Reporting Program (GHGRP) but also funding for more accurate methane emissions measurement. 


Uncertainty about own emissions is also important in other countries where methane regulation is largely nonexistant. Recent advances in remote sensing such as MethaneSAT will lead producers and other stakeholders to have increasingly accurate information about their own lost revenues. On one hand, remote sensing can be good as it should  help operators in these countries detect and eliminate emissions from super-emitter events. On the other hand, our theory and empirical results suggest that satellite monitoring could have an adverse effect on emissions from lower emission sites because satellite monitoring is effectively information provision subject with a threshold $H$ and therefore can displace producers' own leak detection and repair efforts. And because satellites have a high minimum detection threshold, these adverse effects will not necessarily be detectable by satellite. This is a particular policy concern because a large fraction of total wells in the world have both low production and low methane emissions at the well level but together may constitute a significant fraction of methane emissions \citep{bib:omara}. 


\section{Conclusion}
\label{sec:conclusion}

In this paper, we examine how receiving information about a problem affects the firm's decision to fix the problem. In our methane leakage setting, we show that getting the good news of low leakage leads to greater methane emissions at endline, with the likely mechanism being reduced site maintenance effort. In particular, we find that for sites with zero measured leakage at baseline, treatment with information increases emissions by \input{Figures/single_numbers/median_emiss_effect_round.tex}kg per day per treated site, imposing a climate cost of \$\input{Figures/single_numbers/median_emiss_effect_DOLLARS.tex}per day per treated site. The results highlight the conundrum of information provision: While the regulator may wish to provide information about the nature of the problem (such as which specific components are leaking), information provision will also necessarily communicate the severity of the problem (how much methane is leaking). If it turns out that the problem is negligible, information provision can disincentivize effort, making the future problem worse. Our results are novel in that they show that this effect can exist for firms (and not just households as in \citet{bib:costa_kahn} and \citet{bib:castillo_petrie}) and that the effect does not rely on behavioral mechanisms. 

Our results are likely to extend to any oil and gas setting in which profit maximizing firms produce and sell natural gas to the marketplace. One setting in which this does not hold is the case of oil production with associated associated natural gas where there is no nearby pipeline and therefore the gas is flared off rather than sold. In this setting, informing firms of flaring emissions will likely have little effect unless the amount of methane lost from flaring is large enough to justify building a natural gas pipeline. Another setting in which our results are less likely to apply is settings in which decision makers lack the incentives or ability to enact the profit maximizing level of leak detection and repair effort. For example, in Venezuela, the collapse of the oil technocracy in 2016 led to an a significant increase in methane emissions in spite of the country having significant energy shortages \citep{bib:venezuela}. 

Future work is needed that can build on these findings. First, theoretical work is needed for methane emissions contexts to examine how to structure optimal nonpoint source pollution regulation when firms have uncertainty about their own true emissions. Second, as this paper uses a small sample, future work could use larger samples to estimate more precisely the heterogenous effects of methane monitoring information provision, as well as also measure firms' site visit, maintenance, and repair activity. That data could then be used to estimate firms' repair costs, ex-ante beliefs about emissions, and how new policies, such as monitoring and Pigouvian taxes, will affect leak detection, repair, and emissions. Third, work is needed to understand heterogeneity in ability to implement effective leak detection and repair programs. For example, financing leak detection and repair may become more difficult as well production declines. Also, small operators may have more limited budgets and as a result may have greater uncertainty about their own emissions as well as less ability to curtail leaks (see \citet{bib:boomhower} for a related discussion on small firms and environmental liability). Fourth, work is be needed to quantify how recent technological improvements in monitoring -- including monitoring via satellite, aircraft, drone, and tower -- affect firms' leak repair choices and methane emissions as well the extent to which public monitoring crowds out existing self-monitoring by firms. Fifth, work is needed to measure the efficacy of new methane control policies including the newly revised NSPS and the new IRA methane fee in the United States.  


\newpage

\bibliography{References.bib}

\clearpage
\appendix
\setcounter{page}{1}
\renewcommand{\thepage}{A-\arabic{page}}

\begin{centering}
\Huge{\center Incentives and Information in Methane Leak Detection and Repair: \\ Web Appendix}
\vspace{1cm}

\large{\centering \hspace{1cm} Eric K. Lewis \&  Jiayang (Lyra) Wang \& Arvind P. Ravikumar}
\end{centering}
\vspace{1cm}


\section{Proofs for section \ref{sec:model}}
\label{sec:model_extensions}


\paragraph{Proof that $R^{*0}(L)$ is weakly increasing in $L$.} 

To see this, assume that the opposite holds: For two levels of leakage $L_1 < L_2$, assume that $R^{*0}(L_1) > R^{*0}(L_2)$. This therefore implies that $P R^{*0} (L_1) - C_R( R^{*0} (L_1), L_2) > P R^{*0} (L_2) - C_R( R^{*0} (L_1), L_2) \geq P R^{*0} (L_2) - C_R( R^{*0} (L_2), L_2)$, where both inequalities rely on the assumption that $R^{*0}(L_1) > R^{*0}(L_2)$ and the second inequality also relies on the assumption that repair costs are weakly increasing in repair effort $R$. These two inequalities together imply that $R^{*0}(L_2)$ is not an arg max of $P R - C_R(R; L_2)$, which is a contradiction of the definition of $R^{*0}(L)$. Therefore, it must be that $R^{*0}(L)$ is weakly increasing in $L$.

By the same logic, $R^{*1}(L)$ is also weakly increasing in $L$.

\paragraph{Proof of expressions \ref{eq:R_star_inequality}, \ref{eq:R_stars_equal_if_R1_positive}, and \ref{eq:R1_is_zero_if_R0_is_zero}.}

Recall that the definitions of $R^{*0}(L)$ and $R^{*1}(L)$ are as follows:
\begin{align}
R^{*0}(L) & = \max \{x \in [0, L] | \pi(x; L) = \max_{y \in [0, L]} (\pi(y ; L))\} \\
R^{*1}(L) & = \max \{x \in [0, L] | \pi(x; L)+\zeta\cdot 1(x>0) = \max_{y \in [0, L]} (\pi(y ; L)+\zeta\cdot 1(y>0))\}
\end{align}
where:
\begin{align} 
\pi(R; L) & \equiv PR - C_R(R; L) \\
\zeta & \equiv P \delta - C_V < 0
\end{align}

Therefore, the difference between the two is a fixed cost $\zeta$. Therefore, to prove expressions \ref{eq:R_star_inequality}, \ref{eq:R_stars_equal_if_R1_positive}, and \ref{eq:R1_is_zero_if_R0_is_zero}, let $g(x)$ be a continuous function defined on the interval $[0, L]$ and define $f(x ; a)$ as:
\begin{equation}
f(x ; a) = a \cdot 1(x>0) + g(x)
\end{equation}

Then define $x^*(a)$ as being the largest arg max of $f(x; a)$:
\begin{equation}
x^*(a) = \max \{ x \in [0, L] | f(x ; a) = \max_{y \in [0, L]} f(y ; a) \}
\end{equation}

First, we prove expression \ref{eq:R1_is_zero_if_R0_is_zero} by showing that if $x^*(a) = 0$, $x^*(a') = 0 \; \forall a' \leq a$. To see this, assume $x^*(a) = 0$. Then:
\begin{align}
0 + g(0) & > a \cdot 1(x > 0) + g(x) \; \; \forall x \in (0, L] \\
0 + g(0) & > a' \cdot 1(x > 0) + g(x)  \; \; \forall a' \leq a, x \in (0, L]
\end{align} 
where the first line comes from the definition of $x^*(a)$ and the second comes from the fact that $a' \leq a$ and implies that $x^*(a') = 0$ by the definition of $x^*$. The implication for our setting is that if the agent chooses zero repair effort, increasing the fixed costs of repair will also lead to zero repair effort.

Second, we prove expression $\ref{eq:R_stars_equal_if_R1_positive}$ by showing that if $x^*(a) > 0$, then $x^*(a') = x^*(a) \forall a' \geq a$. By the definition of $x^*(a)$, if $x^*(a) >0$, then:
\begin{align}
a + g(x^*(a)) & \geq g(0) \\ 
a + g(x^*(a)) & \geq a + g(x) \; \; \forall x \in (0, x^*(a)] \\
a + g(x^*(a)) & > a + g(x) \; \; \forall x \in (x^*(a), L]
\end{align} 

This implies that for any $a' \geq a$,
\begin{align}
a' + g(x^*(a)) & \geq g(0) \\ 
a' + g(x^*(a)) & \geq a' + g(x) \; \; \forall x \in (0, x^*(a)] \\
a' + g(x^*(a)) & > a' + g(x) \; \; \forall x \in (x^*(a), L]
\end{align} 

Which is the definition of $x^*(a')$, meaning that $x^*(a) = x^*(a')$. The implication for our setting is that if the agent chooses positive repair effort, further decreasing fixed costs will lead to the same level of repair.

These two proofs then imply that for a given leakage level $L$, there are two possible repair choices, one positive, and the other zero. These two proofs also imply that as $a$ decreases (and as the fixed costs of doing any repair increase), repair levels will decrease, therefore proving equation \ref{eq:R_star_inequality}.




\section{Empirical Appendix}
\label{sec:empirical_appendix}

\subsection{Additional Data Background}
\label{sec:data}
The data we use comes from \citet{bib:wang}. Data in that paper was limited to \input{Figures/single_numbers/count_wang_sample.tex}sites where all sites were visited on time -- e.g., where triannual visit sites received all four surveys (baseline survey, first follow-up survey, second follow-up survey, and endline survey) on schedule. We therefore augment the data with \input{Figures/single_numbers/count_added_sample.tex}additional sites where baseline and endline surveys were conducted on time but other surveys were not necessarily conducted on time. We do this to reduce the likelihood that treatment status is correlated with probability of being in the sample. For example, three of the added sites were triannual sites where the surveyors were unable complete the first follow-up visit in November 2018.




\subsection{Operator descriptives and testing SUTVA}
\label{sec:SUTVA}

Figure \ref{fig:operator_count_frac_treated} graphs the total number of sites for each operator (x-axis) as well as the fraction of sites that are treated (y-axis). There is one operator that operates \input{Figures/single_numbers/highest_site_count.tex} sites and two operators that operate one site each. As expected, there is significant variation in the fraction of sites treated for operators that have few sites, but the variance decreases as number of total sites increases.

\begin{figure}[t]
\centering
\includegraphics[width=0.7\linewidth]{Figures/fraction_treated_by_operator.pdf}
\caption{Scatter plot of total number of sites by operator (x-axis) and the fraction of operator sites that are treated (y-axis). Slight perturbation added to points because of cases where multiple operators have the same site count and same fraction of sites treated.} 
\label{fig:operator_count_frac_treated}
\end{figure}

As randomization was at the site level rather than the operator level, we examine whether there is evidence of spillovers, focusing on examining spillovers from treatment sites to control sites. Our regression results are in table \ref{tab:sutva}. There, we regress endline leakage on a series of controls, doing so separately for control sites that have zero baseline leakage (columns 1 and 2) and those that have above-median baseline leakage (columns 3 and 4). Controls include the log of total number of sites in the data (controlling for size differences between operators), the fraction of operator sites that are treated, and the fraction of treatment sites that have above-median baseline leakage (columns 2 and 4 only). We also control for baseline leakage in columns 3 and 4. We do not find statistically significant evidence of spillovers -- neither the coefficient on fraction of operator sites that are treated nor the coefficient on the fraction of treatment sites that have above-median baseline leakage are statistically significant. 

\begin{table}[t]
\input{Figures/spillover_test_control_sites_OLS.tex}
\caption{OLS regressions for control sites to test SUTVA. The dependent variable is endline leakage. Control variables include the log of total number of sites by operator, fraction of operator sites that are treated, fraction of treatment sites that have above-median leakage, and baseline leakage. Regressions are done separately for sites with zero baseline leakage (columns 1 and 2) and sites with above-median baseline leakage (columns 3 and 4). Standard errors are clustered by operator.}
\label{tab:sutva}
\end{table}




\subsection{Balance using CDFs}
\label{app:balance_cdfs}

\begin{figure}[t!]
    \centering
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_baseline_leakage.pdf} 
        \caption{Leakage} \label{fig:baseline_cdf_leakage}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_baseline_emissions.pdf} 
        \caption{Total Emissions} \label{fig:baseline_cdf_emissions}
    \end{subfigure}
    \caption{The cumulative distribution functions (CDFs) of leakage and total emissions at baseline, comparing treatment and control groups. Panel (a) shows the CDFs of baseline leakage; Panel (b) shows the CDFs of total baseline emissions, summing over leakage and venting.}
    \label{fig:baseline_cdf}
\end{figure}

In figure \ref{fig:baseline_cdf_leakage}, we further examine balance by examining the CDF of leakage as measured at baseline, comparing treatment and control sites. Because of the long right tail, we graph the CDF of the log of one plus baseline leakage. We find that the treatment and control distributions are largely similar, as shown in panel (a) of table \ref{tab:balance}. However, we find that control sites are more likely to have zero baseline leakage and less likely to have small but positive leakage. We find that this difference is marginally significant: A test that treatment and control groups are equally likely to have zero baseline leakage has a p value of \input{Figures/single_numbers/baseline_leakage_is_zero_p_test.tex}\unskip. Similar results hold when examining baseline total emissions (summing over leakage and emissions, as shown in figure \ref{fig:baseline_cdf_emissions}. There, we also find that control sites are more likely to have zero baseline emissions, with a p value equal to \input{Figures/single_numbers/baseline_emissions_is_zero_p_test.tex}\unskip.  



\subsection{Power, type S errors, and exaggeration ratios}
\label{sec:gelman_carlin}

In this section, we discuss retrospective calculations of power as well as type S error probabilities and exaggeration ratios following \citet{bib:gelman_carlin}. For this, our analysis focuses on the OLS results in columns 1 and 3 of panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM} where we limit the analysis to those observations that had zero leakage at baseline. This analysis requires us to make assumptions about the true treatment effect, and so this we assume that the true treatment effect is either the estimated treatment effect or some fraction thereof. We then use standard formulas to compute power, and specifically compute power assuming that the standard deviation is unknown and may differ between the treatment and control groups. We also compute two measures from \citet{bib:gelman_carlin}. The first is type S error, which is the probability that the estimated coefficient is the wrong sign, conditional on the estimated coefficient being statistically significant at the $\alpha = 5\%$ level. The second is the exaggeration ratio, which is the expectation of the absolute value of the estimated treatment effect relative to the true effect, conditional on the treatment effect being statistically significant at the $\alpha = 5\%$ level. 

\begin{table}[t]
\input{Figures/gelman_carlin_beyond_power.tex}
\caption{Calculations of power, type-S errors, and exaggeration ratios following \citet{bib:gelman_carlin}. The first 4 rows correspond to the leakage OLS regression in column 1 of panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}; the second 4 rows correspond to the total emissions OLS regression in column 3 of panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}. Estimated treatment effects are given in the 1st column and scaled hypothetical actual treatment effects are given in the 2nd column, where the hypothetical treatment effects are 100\%, 50\%, 25\%, and 10\% of the estimated treatment effect. Standard deviations and observation counts are in the third through sixth columns. The seventh, eighth, and ninth columns give calculated power, type S error probability, and exaggeration ratio assuming the treatment effect given in column 2.}
\label{tab:gelman_carlin_beyond_power.tex}
\end{table}



Our results are in table \ref{tab:gelman_carlin_beyond_power.tex}, where the first 4 rows of table \ref{tab:gelman_carlin_beyond_power.tex} correspond to the leakage regressions in column 1 of panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM} and the last 4 rows correspond to total emissions regressions in column 3 of panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM} (as shown in the first column). For the 1st and 5th row, we examine the case where the true treatment effect is equal to the estimated treatment effect. Then in the remaining rows (rows 2-4 and rows 6-8), we examine the case where the true treatment effects are either 50\%, 25\%, or 10\% of the estimated treatment effect, respectively. This hypothetical treatment effect is shown in the 2nd column. The third and fourth columns include estimates of standard deviation for control and treatment sites, and the fifth and sixth contain the count for control and treatment sites.

Column 7 displays the calculated power, and shows that the experiment is underpowered assuming that the true treatment effect is the estimated treatment effect. The maximum power is 0.51, which is far less than the standard of 0.80. Column 8 displays the probability of type S error. We find that in spite of low power, type S probabilities are fairly low -- on the order of 10\% or less as long as the true treatment effect is at least 25\% the size of the estimated treatment effect -- although they increase to around one-third if the true treatment effect is one-tenth of the estimated treatment effect. Finally, the estimated exaggeration ratio ranges from about 1.5 if the true treatment effect is equal to the estimated treatment effect to 12 or above if the true treatment effect is one-tenth of the estimated treatment effect. 





\subsection{Sites with zero baseline leakage}
\label{sec:zero_baseline_leakage_appendix}

Here, we present additional results on treatment effects for sites with zero baseline leakage as well as the full regression results for the regression results from table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}.

\begin{figure}[t!]
    \centering
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_change_emissions_base_leak_is_zero.pdf} 
        \caption{Absolute change in total emissions} \label{fig:cdf_abs_change_emissions_base_leak_is_zero}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_relative_change_emissions_base_leak_is_zero.pdf} 
        \caption{Relative change in total emissions} \label{fig:cdf_relative_change_emissions_base_leak_is_zero}
    \end{subfigure}
    \caption{The cumulative distribution functions (CDFs) of change in emissions for sites with zero baseline leakage, comparing treatment and control. Panel (a) shows the CDFs of the absolute change in total emissions (endline minus baseline). Panel (b) shows the CDFs of the relative change in emissions ((endline minus baseline)/baseline.)}
    \label{fig:cdf_change_emissions_base_leak_is_zero}
\end{figure}



Figure \ref{fig:cdf_change_emissions_base_leak_is_zero} graphs the cdfs of the \emph{change} in total emissions for treatment and control sites, conditional on zero leakage at baseline. Panel (a) graphs the change in absolute emissions (baseline minus endline). There, we find that the treatment group cdf tends to be to the right of the control group cdf for most of the distribution. However, the treatment cdf is slightly to the left for the lowest values of the change in emissions. This reflects that in cases where baseline leakage was zero but baseline venting was high, operators responded to the information treatment by reducing venting. Panel (b) graphs the relative change in total emissions ((baseline minus endline)/baseline). There, we find that the treatment group CDF largely first-order stochastically dominates the control group CDF, consistent with the idea that treatment with information led to a relative increase in endline emissions.

Table \ref{tab:reg_zero_baseline_leakage_OLS_detail} presents the full regression results for the OLS specification in panel (a) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}. Table \ref{tab:reg_zero_baseline_leakage_TWOPM_detail} presents the full results for the two-part model specifications in panel (b) of table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM}. For column 4 of the two-part model in table \ref{tab:reg_zero_baseline_leakage_TWOPM_detail}, we only include the full set of control variables for the GLM specification (but not the logit specification) because of the few observations that have zero baseline emissions. The missed bootstrap reps come from bootstrap draws of the data where there are no cases where the outcome variable is equal to zero. For all tables, standard errors are clustered by operator. For the two-part model results, confidence intervals are calculated using \input{Figures/single_numbers/bs_rep_count.tex}bootstrap draws, clustered by operator.

\begin{table}[p]
\input{Figures/reg_zero_baseline_leakage_OLS_detail.tex}
\caption{OLS regression results for sites with zero baseline leakage, giving the full regression results corresponding to table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM} panel (a). Columns 1 and 2 examine the effect of treatment on endline leakage; columns 3 and 4 examines the effect of treatment on endline total emissions. Controls added in columns 2 and 4 include the treatment indicator, log(1 + baseline venting), and the interaction of treatment with log(1 + baseline venting). Baseline venting is measured in kilograms per day. Standard errors are clustered by operator.}
\label{tab:reg_zero_baseline_leakage_OLS_detail}
\end{table}

\begin{table}[p]
\input{Figures/reg_zero_base_leakage_TWOPM_detail.tex}
\caption{Two-part model regression results for sites with zero baseline leakage, giving the full regression results corresponding to table \ref{tab:reg_zero_baseline_leakage_OLS_TWOPM} panel (b). Columns 1 and 2 examine the effect of treatment on endline leakage; columns 3 and 4 examines the effect of treatment on endline total emissions. Controls added in columns 2 and 4 include the treatment indicator, log(1 + baseline venting), and the interaction of treatment with log(1 + baseline venting). Baseline venting is measured in kilograms per day. Standard errors are clustered by operator. Confidence intervals for average treatment effects are calculated using \protect\input{Figures/single_numbers/bs_rep_count.tex}bootstrap draws, clustered by operator.}
\label{tab:reg_zero_baseline_leakage_TWOPM_detail}
\end{table}

 
 

Table \ref{tab:Treat_x_Vent} gives OLS robustness results for sites with zero baseline leakage where we interact treatment with baseline levels of venting. We find that the coefficient on the interaction is positive but statistically insignificant.

\begin{table}[t]
\input{Figures/reg_zero_baseline_leakage_OLS_Treat_x_Vent.tex}
\caption{Regression results for sites with zero baseline leakage. Column 1 examine the effect of treatment on endline leakage; column 2 examines the effect of treatment on endline total emissions. Controls include the treatment indicator, log(1 + baseline venting), and the interaction of treatment with log(1 + baseline venting). Baseline venting is measured in kilograms per day.}
\label{tab:Treat_x_Vent}
\end{table}

Table \ref{tab:count_reg} gives robustness results for sites with zero baseline leakage where we use the count of leaks rather than the total emissions from leaks as the outcome variable. Columns 1 and 2 use OLS; columns 3 and 4 use Poisson. For all specifications, we find that the coefficient on treatment is positive but statistically insignificant.

\begin{table}[t]
\input{Figures/reg_zero_baseline_leakage_COUNT.tex}
\caption{Regression results for sites with zero baseline leakage where the dependent variable is the count of leaking components at endline. Columns 1 and 2 use OLS; columns 3 and 4 use Poisson regression. Controls in columns 2 and 4 include the treatment indicator, log(1 + baseline venting), and the interaction of treatment with log(1 + baseline venting). Baseline venting is measured in kilograms per day.}
\label{tab:count_reg}
\end{table}


\subsubsection{Treatment intensity}
\label{sec:treatment_intensity}

Because of power issues, our analysis largely focuses on comparing treatment and control groups and abstracts away from different intensities of treatment (annual vs. biannual vs. triannual treatment groups). Here, we briefly explore treatment intensity, focusing on the effect of treatment intensity on endline leakage and endline emissions for sites with zero baseline leakage (extending the results in section \ref{sec:zero_baseline_leakage})

Table \ref{tab:reg_zero_baseline_leakage_treatment_intensity_OLS} is similar to table \ref{tab:reg_zero_baseline_leakage_OLS_detail} except that it adds in to all specifications an additional variable, treatment intensity, which takes the value of zero for control and annual treatment sites, one for biannual treatment sites, and two for triannual treatment sites. We find that in all specifications, the coefficient on treatment intensity is statistically insignificant from zero. We also find that in all specifications, the coefficient on treatment is positive but not statistically significant from zero. 

\begin{table}[p]
\input{Figures/reg_zero_baseline_leakage_treatment_intensity_OLS.tex}
\caption{Estimates of treatment effects using OLS, conditional on zero baseline leakage. The treatment intensity variable takes the value of zero for control sites and annual treatment sites, one for biannual treatment sites, and two for triannual treatment sites.Columns 1 and 2 examine the effect of treatment on endline leakage; columns 3 and 4 examine the effect of treatment on endline total emissions. Standard errors are in parentheses and are clustered by operator. Controls included in columns 2 and 4 are the natural log of one plus baseline venting (in kilograms per day), the natural log of one plus baseline oil production (in cubic meters per day), and the natural log of one plus baseline natural gas production (in thousands of cubic meters per day).}
\label{tab:reg_zero_baseline_leakage_treatment_intensity_OLS}
\end{table}


\subsection{Sites with above-median baseline leakage}
\label{sec:above_median}


To what extent did treatment reduce leakage and total emissions at sites with high baseline leakage? While our results in figures \ref{fig:lpoly_log1_leakage_on_log1_leakage_DIF} and \ref{fig:lpoly_log1_emissions_on_log1_leakage_DIF} suggest a statistically significant treatment effect for the highest baseline leakage places, these results may be driven by out-of-sample extrapolation (because the maximum value of baseline leakage for the treatment group is higher than the maximum value of baseline leakage for the control group -- see figures \ref{fig:lpoly_log1_leakage_on_log1_leakage} and \ref{fig:lpoly_log1_emissions_on_log1_leakage}). 

We next turn to analyzing the overall treatment effects for sites where baseline leakage is above the median level. We focus on this case because figure \ref{fig:baseline_cdf_leakage} shows that the CDFs of baseline leakage for treatment and control groups most closely track each other when baseline leakage is above the median. We graph the CDFs of both treatment and control endline leakage in figure \ref{fig:cdf_endline_leakage_base_leak_is_above_median}. There, we find that the CDFs of treatment and control groups are largely similar to each other, with no first-order stochastic dominance. We find very similar results when graphing the treatment and control CDFs of endline total emissions in figure \ref{fig:cdf_endline_emissions_base_leak_is_above_median}.  The fact that the treatment and control CDFs are largely similar implies that any estimated treatment effect is unlikely to be significantly different from zero, and any estimated treatment effects that are statistically significant are likely to be driven by functional form and outlier values.  

\begin{figure}[ph]
    \centering
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_endline_leakage_base_leak_is_above_median.pdf} 
        \caption{Leakage} \label{fig:cdf_endline_leakage_base_leak_is_above_median}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_endline_emissions_base_leak_is_above_median.pdf} 
        \caption{Total Emissions} \label{fig:cdf_endline_emissions_base_leak_is_above_median}
    \end{subfigure}
    \caption{The cumulative distribution functions (CDFs) of leakage and total emissions at endline, conditional on greater than median levels of leakage at baseline, comparing treatment and control groups. Panel (a) shows the CDFs of endline leakage. Panel (b) shows the CDFs of total endline emissions, summing over leakage and venting.}
    \label{fig:endline_cdf_base_leak_is_above_median}
\end{figure}

\begin{figure}[ph]
    \centering
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_endline_leakage_base_leak_is_below_median_above_zero.pdf} 
        \caption{Leakage} \label{fig:cdf_endline_leakage_base_leak_is_below_median_above_zero}
    \end{subfigure}
    \hfill
    \begin{subfigure}[t]{0.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{Figures/cdf_endline_emissions_base_leak_is_below_median_above_zero.pdf} 
        \caption{Total Emissions} \label{fig:cdf_endline_emissions_base_leak_is_below_median_above_zero}
    \end{subfigure}
    \caption{The cumulative distribution functions (CDFs) of leakage and total emissions at endline, conditional on leakage at baseline being greater than zero but less than the median, comparing treatment and control groups. Panel (a) shows the CDF of leakage. Panel (b) shows the CDF of total emissions.}
    \label{fig:endline_cdf_base_leak_is_below_median_above_zero}
\end{figure}


\newpage

\subsection{Sites with non-zero and below-median baseline leakage}

In contrast with sites with zero baseline leakage (\input{Figures/single_numbers/count_zero_baseline_leakage.tex}observations, see section \ref{sec:zero_baseline_leakage}) and sites with above-median baseline leakage (\input{Figures/single_numbers/count_baseline_leakage_above_median.tex}observations, see the above appendix section \ref{sec:above_median}), there are relatively few sites with baseline leakage that is above zero but below the median (\input{Figures/single_numbers/count_baseline_leakage_below_median_above_zero.tex} observations). Of those, only 4 are control sites, and there is a lack of balance between treatment and control in total emissions. Therefore, we are hesitant to draw any conclusions about treatment effects for those sites. Nevertheless, we present some CDFs describing differences between treatment and control sites. See figure \ref{fig:endline_cdf_base_leak_is_below_median_above_zero}, which is the CDFs of endline leakage (panel a) for treatment and control groups and the CDF of endline emissions (panel b) for treatment and control groups, all conditional on baseline leakage being above zero but below the median level of baseline leakage. 



\subsection{Repair status}
\label{sec:repairs}

Repair activity was only recorded for leaking components at treatment sites where the site surveyors could physically tag components and tags were not lost nor removed.\footnote{Some components could not be tagged because components were not within reach or because they were not safe to physically tag. Some leaking components that were tagged were not tracked over time, either because the tags were lost or because operators removed the tags.} Unfortunately, of the \input{Figures/single_numbers/repair_count_all_leaks.tex} leaking components we observe at treatment sites at baseline, only \input{Figures/single_numbers/repair_count_leaks_tracked.tex} could be tracked. Repairs were identified from visually inspecting the component for repair. We find that \input{Figures/single_numbers/repair_percent_repaired.tex}\unskip\% of the \input{Figures/single_numbers/repair_count_leaks_tracked.tex}tracked leaked components are repaired by the follow-up survey. 

We examine the probability that a leaking component is repaired as a function of initial leakage volume at baseline, total leakage from all other components at baseline at the site, and total venting at baseline from the site. We note that baseline leakage, total other leakage, and total venting for the site are not exogenous and the coefficients should not be interpreted as causal. In our regressions, we include controls for type of leak (such as whether valve, pneumatic, or flange). We also include an indicator variable for whether the site is 20+ years old (site establishment year is 1998 or before) as well as indicator variables for biannual and triannual treatment.\footnote{We include biannual and triannual treatment controls because triannual sites had a shorter time until first follow-up survey than biannual sites, which in turn had a shorter time until first follow-up survey than annual sites. Therefore, treatment category may be correlated with likelihood that repair was observed. Both because of this and because selection into this sample may be correlated with treatment, we do not interpret the coefficients on these as causal.} Our estimates are in table \ref{tab:reg_repair_status}. Column 1 controls only for leakage and venting amounts, column 2 adds in a control for whether the site is 20+ years old, and column 3 adds in controls for treatment frequency and component-type fixed effects.

In all three columns, we find that repair probability is increasing in own-component leakage, with the coefficient statistically significant, suggesting that firms are more incentivized to repair leaks when leaks are larger. For total leakage from all other components at the site, we find that total other leakage is negatively correlated with repair, opposite of what the stylized model would predict. This negative correlation is potentially explained by either the selected sample or omitted variable bias. For example, sites with higher site visit costs and higher average leak repair costs will tend to have both higher leakage rates at baseline as well as lower likelihood of repair at later dates. Finally, for total venting, we find that higher venting is correlated with higher likelihood of repair, although the correlation is only statistically significant in the first specification. These results are consistent with the idea that the information treatment helped firms learn about high venting volumes and therefore incentivized these firms to make site visits not only to reduce venting but also to repair leaks. 


\begin{table}[t]
\input{Figures/component_repair_regressions.tex}
\caption{OLS regressions using component-level data on leaks where the dependent variable is an indicator for whether the leak was repaired by the first follow-up survey after the baseline survey. Sample limited to leaking components at treatment sites that could be tracked over time. Standard errors are clustered by operator. Controls in column 3 are fixed effects for each component type.}
\label{tab:reg_repair_status}
\end{table}


\end{document}


